posted 12-13-2007 10:53 AM            

Buster:

quote:

---

I beleive it is multi-facet but my mentor doesn't like that term. Once you deviate from the same question rephrased- I was trained its multi issue.

R6 Did you knowingly give that female any of that money?

R9 Regarding that money did you take it?

R12 Do you know that females identity?

I know they are a little odd, but so is the situation.

---

This is not set in stone, but to clarify concepts:

• Single issue = one conceptual behavioral issue which can be questioned different ways - did one, did them all
  
• Event specific = a term used throughout the NRC/NAS report - It seems the NRC/NAS committee felt the language could obscure the point if we're not careful - I believe they are correct. The historical term seems to have been "specific issue" or "specific," and the confusion arises when we conduct single issue exams in the absence of a known allegation - such as when completing a break-down test or follow-up test on a single issue after an unresolved screening test. Empirically, that is still a screening exam, if there is no known even, no allegation, or no particular reason to suspect involvement. Screening tests are fishing trips. The empirical and mathematical principles that that pertain to diagnostic exams are different (more on that later). So the term "Event specific" forces a clearer understanding and requirement that their is a known event or known allegation before we regard an exam as a diagnostic type test. This is consistent with the successive-hurdles approach that is common in medicine.
  
• multi-facet = a single issue or event of concern, usually a known event or known allegation, for which a test subject might be questioned about various forms of involvement (roles) (e.g., did you rob the first bank, did you shoot the teller, did you drive the getaway car, were you present during that bank robbery, etc., whatever). DI to any of the questions would indicate involvement in the single event or allegation.
  
• mixed-issues = several distinct issues, for which the examinee could conceivably lie about involvement in one or more, while being truthful regarding others. I've heard Don Krapohl use the term "multi-issue" for these exams, but I believe that is too easily confused with multi-facet. So, I prefer the term "mixed-issues," and that is what you'll read throughout OSS-3 documentation. The language is less important than the concept.
  

Basically a multi-facet exam is a mixed-issues exam regarding a single known incident or known allegation for which we have some reason to suspect our subject's involvement, and for which we question/examine the subject regarding different forms or roles of involvement. Ergo, a multi-facet examination regarding a single incident or allegation of concern.

The mathematical principles that pertain to multi-facet exams differ from those of mixed-issues exams. (more later.)

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-13-2007 11:49 AM            

eg.

Did you take that vehicle?
Did you take the vehicle Tuesday?
(EC) Can you take me to that vehicle?

This is multi issue according to my training.

His single issue is this:

Did you take that vehicle?
RE: That vehicle, DY take it?
Are you telling me the truth about not taking that vehicle?

posted 12-13-2007 12:07 PM            

he did it
he did it
he did it

or

he didn't
he didn't
he didn't

Its important that our work adhere so some form of conceptual logic.

However, human psychology and human physiology are far too fuzzy (IMHO) to fit neatly into our little linguistic ven-diagrams.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-13-2007 12:29 PM            

Its not the logic of the language that defines a test as event specific, multi-facet or mix-issues.

It is the logic of the behavior that makes them different.

It is the presence or absence of a known incident with reason to suspect our subject's involvement, or known allegation that defines an exam as a diagnostic test or event specific test.

Diagnostic tests can be single issue or multi-facet.

In the absence of a known event with reason to suspect involvement, or known allegation, a test (any form of test) is by definition a screening test (fishing expedition).

The empirical meanings differ for the results of screening and diagnostic tests - because their construction differs, and the mathematical/statistical models differ.

(more in a bit)

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-13-2007 02:36 PM            

Here is a link to a comparison of OSS-3 results with the training sample - with and without Bonferonni correction to the alpha.

Bonferonni, as explained last night, is the correction to the mathematically inevitable inflation of alpha when completing multiple simulataneous significance test (spot scoring rule in ZCT exams, and MGQT scoring rules), and corrects for the increased likelihood of false positive results due to the addition-rule for deptendent probability events.
http://www.oss3.info/OSS-3_Training_sample_w_and_wo_Bonferonni.pdf

You can see their is a trade-off, with a significant reduction in FPs from 10.4% to 4.8% (p = .006) when we apply Bonferonni correction to the alpha of .05. The trade off of increased inconclusives, from 1.4% to 4.5% (p = <.001). 4.5% is still pretty low, and these figures pretty low so we're not concerned about the INC rate. FPs over 10% are a problem. Bonferonni works well. Also, these figures are with the training sample, which is a theoretical best fit, since we used this same data for bootstrap training of the OSS-3 model. We wouldn't expect to see these low INC rates generalize to other data or live field situations, but this is what he have with the study data.

Now, below is are the OSS-3 results with a 2nd validation sample of mixed formate cases from the DACA/DoDPI archive.

These are pre-2002 exams. This one-page document compares the efficiency of the Senter/2-stage rules to traditional MGQT/spot rules with the multi-facet exams.
http://www.oss3.info/OSS-3_mixed_format_validation_validation_sample.pdf

We've choosen to discuss the results in traditional terms of sensitivity (to deception) and specificity (to truthfulness), because these terms are easily understood by non-polygraph scientists. Sensitivity and specificity are essentially accuracy rates with inconclusives for truthful and deceptive subjects.

You can see the obvious advantages of the Senter/2-stage rules, with the mixed format cases. This difference would be more obscured if we chose to illustrate the data in terms of accuracy without inconclusives.

While the MGQT/spot rules provide good sensitivity to deception, they offer meager specificity compared to the healthy balance that the 2-stage rules achieve. This is consistent with Senter's work from 2003 and 2005.

Because the results of a diagnostic test become a basis for action (e.g., further investigation, prenatal care, medications, surgery, right-altering decisions, etc.), they must provide adequate specificity.

MGQT rules are a problem in mult-facet exams. Multi-facet exams are known-incident exams, and diagnostic tests must provide adequate specificity - to assure we can rule-out people who are not involved in or do not express the condition or issue of concern.

For this reason, OSS-3 uses 2-stage rules for multi-facet exams.

So, Senter rules prevail with both single issue and multi-facet event specific exams. However, screening exams are often mixed issues - what do we do about that.

We have decision rules specifically for screening exams - using an omnibus nonparametric ANOVA (KW-test) that is optimized for sensitivity. Omnibus statistics are simply formulae intended to test everything at once - they eliminate the effect of the addition rule on inflated alpha and increased errors and INCs.

Here is a link to a validation study of OSS-3 using a sample of LEPET screening exams from Texas.
http://www.oss3.info/OSS-3_screening_validation.pdf

Again you can see the inadequate specificity of the spot rules once again. You can also see the reduction in INCs, though we would caution against expecting INCs that low in live field situations.

Because screening test results are used differently than the results of diagnostic tests, we also have different priorities for our accuracy considerations. In screening contexts, negative results mean we're done - further action ceases (e.g., meds, surgery, investigation, testing, whatever.) So what we really value is adequate sensitivity (to deception), with the understanding that we will tollerate some FPs and clean them up at later stages of our diagnostic work.

For that reason, is it both acceptable and common in screening contexts to not include a Bonferonni correction to the inflated alpha.

Here is a link to the result of the LEPET sample, using the sreening rules with and without Bonferonni correction.
http://www.oss3.info/OSS-3_LEPET_screening_w_and_wo_Bonferonni.pdf

Consistent with our theoretical expectations, and experience in other testing and research environment, you can see that our screening test is better, more accurate with fewer false negatives, without the use of Bonferonni. While the p-values are not statistically significant, it is still acceptable to so everything we can to optimize the test for its intended use.

Someone (Barry) should fact-slap the Sergeant1107 with the idea that when we are talking about at test whose accuracy is already well above chance, all improvements will be modest, even modest improvements are important (and sometimes hard to achieve).

OK, any questions?

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-13-2007 02:41 PM            

Ray, you are a talent and an asset to the field. We should get CEU's for reading your posts.